Search for: All records

Transfer learning from the model trained on large datasets to customized downstream tasks has been widely used as the pre-trained model can greatly boost the generalizability. However, the increasing sizes of pre-trained models also lead to a prohibitively large memory footprints for downstream transferring, making them unaffordable for personal devices. Previous work recognizes the bottleneck of the footprint to be the activation, and hence proposes various solutions such as injecting specific lite modules. In this work, we present a novel memory-efficient transfer framework called Back Razor, that can be plug-and-play applied to any pre-trained network without changing its architecture. The key idea of Back Razor is asymmetric sparsifying: pruning the activation stored for back-propagation, while keeping the forward activation dense. It is based on the observation that the stored activation, that dominates the memory footprint, is only needed for backpropagation. Such asymmetric pruning avoids affecting the precision of forward computation, thus making more aggressive pruning possible. Furthermore, we conduct the theoretical analysis for the convergence rate of Back Razor, showing that under mild conditions, our method retains the similar convergence rate as vanilla SGD. Extensive transfer learning experiments on both Convolutional Neural Networks and Vision Transformers with classification, dense prediction, and language modeling tasks show that Back Razor could yield up to 97% sparsity, saving 9.2x memory usage, without losing accuracy. The code is available at: https://github.com/VITA-Group/BackRazor_Neurips22. 

Purpose There is recent emphasis on designing new materials and alloys specifically for metal additive manufacturing (AM) processes, in contrast to AM of existing alloys that were developed for other traditional manufacturing methods involving considerably different physics. Process optimization to determine processing recipes for newly developed materials is expensive and time-consuming. The purpose of the current work is to use a systematic printability assessment framework developed by the co-authors to determine windows of processing parameters to print defect-free parts from a binary nickel-niobium alloy (NiNb5) using laser powder bed fusion (LPBF) metal AM. Design/methodology/approach The printability assessment framework integrates analytical thermal modeling, uncertainty quantification and experimental characterization to determine processing windows for NiNb5 in an accelerated fashion. Test coupons and mechanical test samples were fabricated on a ProX 200 commercial LPBF system. A series of density, microstructure and mechanical property characterization was conducted to validate the proposed framework. Findings Near fully-dense parts with more than 99% density were successfully printed using the proposed framework. Furthermore, the mechanical properties of as-printed parts showed low variability, good tensile strength of up to 662 MPa and tensile ductility 51% higher than what has been reported in the literature. Originality/value Although many literature studies investigate process optimization for metal AM, there is a lack of a systematic printability assessment framework to determine manufacturing process parameters for newly designed AM materials in an accelerated fashion. Moreover, the majority of existing process optimization approaches involve either time- and cost-intensive experimental campaigns or require the use of proprietary computational materials codes. Through the use of a readily accessible analytical thermal model coupled with statistical calibration and uncertainty quantification techniques, the proposed framework achieves both efficiency and accessibility to the user. Furthermore, this study demonstrates that following this framework results in printed parts with low degrees of variability in their mechanical properties. 

Symmetries crucially underlie the classification of topological phases of matter. Most materials, both natural as well as architectured, possess crystalline symmetries. Recent theoretical works unveiled that these crystalline symmetries can stabilize fragile Bloch bands that challenge our very notion of topology: Although answering to the most basic definition of topology, one can trivialize these bands through the addition of trivial Bloch bands. Here, we fully characterize the symmetry properties of the response of an acoustic metamaterial to establish the fragile nature of the low-lying Bloch bands. Additionally, we present a spectral signature in the form of spectral flow under twisted boundary conditions.